Spatial Language Understanding with Multimodal Graphs using Declarative Learning based Programming

نویسندگان

  • Parisa Kordjamshidi
  • Taher Rahgooy
  • Umar Manzoor
چکیده

This work is on a previously formalized semantic evaluation task of spatial role labeling (SpRL) that aims at extraction of formal spatial meaning from text. Here, we report the results of initial efforts towards exploiting visual information in the form of images to help spatial language understanding. We discuss the way of designing new models in the framework of declarative learning-based programming (DeLBP). The DeLBP framework facilitates combining modalities and representing various data in a unified graph. The learning and inference models exploit the structure of the unified graph as well as the global first order domain constraints beyond the data to predict the semantics which forms a structured meaning representation of the spatial context. Continuous representations are used to relate the various elements of the graph originating from different modalities. We improved over the state-of-the-art results on SpRL.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning

We propose a general approach to modeling semisupervised learning (SSL) algorithms. Specifically, we present a declarative language for modeling both traditional supervised classification tasks and many SSL heuristics, including both well-known heuristics such as co-training and novel domainspecific heuristics. In addition to representing individual SSL heuristics, we show that multiple heurist...

متن کامل

FACTORIE: Probabilistic Programming via Imperatively Defined Factor Graphs

Discriminatively trained undirected graphical models have had wide empirical success, and there has been increasing interest in toolkits that ease their application to complex relational data. The power in relational models is in their repeated structure and tied parameters; at issue is how to define these structures in a powerful and flexible way. Rather than using a declarative language, such...

متن کامل

kLogNLP: Graph Kernel-based Relational Learning of Natural Language

kLog is a framework for kernel-based learning that has already proven successful in solving a number of relational tasks in natural language processing. In this paper, we present kLogNLP, a natural language processing module for kLog. This module enriches kLog with NLP-specific preprocessors, enabling the use of existing libraries and toolkits within an elegant and powerful declarative machine ...

متن کامل

The Efficacy of Procedural and Declarative Learning Strategies on EFL Students’ Oral Proficiency

Style and strategies in EFL learning contexts and the effects of task types were explored to enhance language learning strategies. Using a quantitative pre-test, post-test design and interviews, this study investigated the effects of procedural and declarative learning strategies on EFL learners’ acquisition of English past tense performing narrative tasks. The participants were 396 male and fe...

متن کامل

PRISM Revisited: Declarative Implementation of a Probabilistic Programming Language Using Delimited Control

PRISM is a probabilistic programming language based on Prolog augmented with primitives to represent probabilistic choice. PRISM is implemented using a combination of low level support from a modified version of B-Prolog, source level program transformation, and libraries for probabilistic inference and learning implemented in the imperative language C. More recently, developers of probabilisti...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2017